UNCLASSIFIED 


DSTO-GD-0734 


18.  Technical  Risk  Analysis  -  Exploiting  the  Power  of 

MBSE 


11  2 

Despina  Tramoundanis  ,  Wayne  Power  and  Daniel  Spencer 
1  2 
Weapons  Systems  Division,  DSTO  and  Aerospace  Concepts 


Abstract 

In  his  2003  review  into  Defence  procurement,  Kinnaird  recommended  that  for  new 
acquisitions  Defence  undertake  a  ' comj)rehensive  analysis  of  technology,  cost  and  schedule  risks' 
and  that  'Government  needs  to  he  assured  that  adequate  scrutiny  is  undertaken  ....by  DSTO  on 
technology  feasibility,  maturity  and  overall  technical  risk'.  As  a  result,  DSTO  performs  Technical 
Risk  Assessments  (TRA)  to  inform  major  acquisition  decisions  during  the  Requirements  phase 
of  the  Capability  Development  process. 

Instructions  for  preparing  the  TRA  are  found  in  the  Technical  Risk  Assessment  Handbook 
(TRAH)i7.  These  instructions  provide  useful  guidance  on  the  nature  of  technology  and 
technical  risks  and  means  for  risk  discovery  and  assessment. 

The  current  TRA  development  practice  has  several  shortcomings,  including: 

•  Existing  templates  do  not  necessarily  fit  every  type  of  acquisition  project. 

•  At  the  early  stages  of  capability  definition,  before  a  materiel  solution  has  been 
selected,  system  decomposition  is  not  always  possible. 

•  The  level  of  discipline  and  rigour  applied  to  risk  analysis  is  variable  depending  on  the 
skills  of  individuals. 

•  System  integration  risk  does  not  receive  adequate  coverage. 

•  The  TRA  is  a  stand-alone  document  meaning  that  the  risk  analysis  is  not  necessarily 
integrated  with  the  capability  definition. 

•  It  is  not  easy  to  see  how  risks  in  one  part  of  the  system  impact  risks  in  other  parts  of 
the  system  that  may  be  directly  or  indirectly  coupled. 

To  address  several  of  these  shortcomings,  this  paper  introduces  the  concept  of  Functional  Risk 
Analysis  (FRA)  conducted  within  a  Model  Based  Systems  Engineering  (MBSE)  environment. 
FRA  is  a  rigorous  technique  used  to  explore  potential  effects  of  functional  failures  or 
degradation  that  result  from  insufficient  technical  readiness,  both  within  and  between  parts  of 
a  system  and  across  system  interfaces.  (FRA  is  analogous  to  Functional  Hazard  Analysis,  a 
technique  applied  in  the  aerospace  domain.)  The  underlying  method  of  FRA  uses  an 
Enhanced  Functional  Flow  Block  Diagram  (EFFBD)  representation  of  the  system  functionality 
and  follows  the  following  procedure: 

1.  Perform  the  following  steps  on  each  function  in  turn: 

a.  Define  the  purpose  and  behaviour  of  the  function. 

b.  Consider  the  technologies  inherent  in  the  function  and  the  potential  failure 
modes  that  may  result  based  on  an  understanding  of  the  technology  readiness. 
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c.  Represent  functional  failure  modes  within  MBSE  model. 

2.  Simulate  or  interrogate  the  functional  model  to  assess  the  potential  impact  of 
functional  failures  on  downstream  functions  and  guide  detailed  system  analysis. 

3.  Record  in  the  MBSE  model  the  identified  risks  (i.e.  the  potential  effect  in  terms  of 
severity  and  probability  of  occurrence). 

Once  the  physical  system  has  been  designed  or  selected,  the  FRA  procedure  can  be  repeated 
using  the  system  architecture  to  assess  and  explore  the  effects  of  component  failures  or 
degradation  that  result  from  insufficient  system  readiness.  The  results  of  the  FRA  are  recorded 
in  the  MBSE  model  from  which  the  TRA  report  is  auto-generated  via  the  running  of  scripts. 
This  paper  will  use  a  generic  weapon  system  example  to  illustrate  the  FRA  technique. 

Presenter  Biography 

Despina  Tramoundanis  was  a  Royal  Australian  Air  Force  Armaments  Engineer  for  20  years 
before  joining  DSTO's  Weapons  Systems  Division.  She  is  currently  the  S&T  advisor  for  a 
Ground-Based  Air  and  Missile  Defence  project.  Her  current  research  interests  include 
development  of  the  WhoIe-of-System  Analytical  Framework,  a  Model-Based  Capability 
Engineering  methodology  for  the  provision  of  cross-Defence  modeling,  simulation,  analysis 
and  Capability  Development  activities.  She  holds  a  Bachelor  of  Engineering  (Chemical)  from 
Monash  University,  an  MSc  in  Explosives  Engineering  from  Cranfield  University  (UK),  a 
Master  of  Defence  Studies  from  UNSW  and  a  Master  of  Defence  Operations  Research  from 
UNSW. 

Wayne  Power  graduated  with  honours  from  the  Queensland  University  of  Technology  (QUT) 
with  a  Bachelor  of  Engineering  (Aerospace  Avionics),  minor  in  Systems  Engineering.  He  has 
spent  the  last  six  years  working  in  Weapons  Capability  Analysis  within  DSTO's  Weapons 
Systems  Division  (WSD).  His  work  in  WSD  has  included  weapon  system  integration 
modelling  and  analysis,  but  the  major  focus  of  his  work  has  revolved  around  researching  and 
developing  the  WhoIe-of-System  Analytical  Framework  (WSAF).  The  WSAF  employs  a 
Model-Based  Systems  Engineering  approach  for  the  provision  of  cross-Defence  modelling, 
simulation,  analysis  and  Capability  Development  activities. 

Daniel  Spencer  works  as  a  systems  engineer  for  Aerospace  Concepts  Pty  Ltd.  He  has  over  a 
decade  of  experience  in  design  and  development  of  systems  solutions  across  a  broad  range  of 
industries,  both  in  Australia  and  the  United  Kingdom.  Dan  holds  a  Bachelor  of  Engineering  in 
Information  Technology  and  Telecommunications  from  the  University  of  Adelaide.  He  has 
been  working  with  Australian  Defence  clients  developing  and  refining  tools  and  methods  for 
a  repeatable  and  comprehensive  MBSE  method,  while  using  this  approach  for  real-world 
capability  definition  and  development  projects. 


210 


UNCLASSIFIED 


UNCLASSIFIED 


DSTO-GD-0734 


Presentation 


UNCLASSIFIED  -  For  Public  Release 

1  Australian  Government 


**  Department  of  Defence 
Defence  Science  and 
Technology  Organisation 


Technical  Risk  Analysis: 
Exploiting  the  Power  of 
MBSE 

Despina  Tramoundanis,  DSTO  (WSD) 
Wayne  Power,  DSTO  (WSD) 

Daniel  Spencer,  Aerospace  Concepts  P/L 


UNCLASSIFIED  -  For  Public  Release 


UNCLASSIFIED  -  For  Public  Release 


Overview 

•  Brief  background 

•  The  need 

•  What  is  Functional  Risk  Analysis  (FRA)? 

•  FRA  Implementation  in  an  MBSE  environment 

•  An  example 
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Kinnaird  (2003): 


For  new  acquisition  Defence  should  undertake  a 

comprehensive  analysis  of  technology,  cost 
and  schedule  risks’ 

‘Government  needs  to  be  assured  that  adequate 
scrutiny  is  undertaken  ...by  DSTO  on  technology 
feasibility,  maturity  and  overall  technical  risk’. 
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Technical  Risk  Assessment 


Pre-1 Pass:  TRI 

P^-Pass  &  2"*^  Pass:  TRA 

Technical  Risk  Assessment  Handbook  (TRAH) 

TRA  templates 

Based  on 

■  Technical  Readiness  Levels  (TRLs) 

■  Risk  assessment  matrix 
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Shortcomings 

•  TRA  templates  do  not  fit  every  type  of  acquisition 

•  Work  only  with  materiel  solutions 

•  Quality  depends  on  the  skills  of  individuals 

•  Inadequate  analysis  of: 

■  System  integration  risk 

■  Risk  coupling 

•  TRA  is  a  stand-alone  document 


UNCLASSIFIED  -  For  Public  Release 


UNCLASSIFIED  -  For  Public  Release 


The  Need 


A  rigorous  technique  to  explore  the  potential  effects 
of  functional  failures  and  performance  degradation 
that  result  from  insufficient  technical  readiness, 
both  within  and  between  parts  of  a  system  and 
across  system  interfaces 
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What  is  FRA? 


A  rigorous  technique  used  with  an  MBSE 
methodology  to  explore  the  potential  effects  of 
functional  failures  and  performance  degradation 
that  result  from  insufficient  technical  readiness  of  a 
system  and  its  interfaces 


Application  of  Functional  Hazard  Assessment  methods  to  risk  analysis 
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Functional  Hazard  Assessment 

(modified  from  SAE  ARP4761) 
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FRA  Procedure  Overview 


•  Commence  with  a  functional  decomposition  of  the 
capability  system  and  include  the  system  interfaces. 

•  Define  the  purpose  and  behaviour  of  each  system 
function. 

•  Consider  potential  failure  modes  of  each  function  eg 
loss  or  degradation  of  function 

•  Determine  the  effect  of  each  failure  on  system 
function  and  operational  /  mission  outcomes 

•  Identify,  analyse  and  record  the  risks  (impact  and 
likelihood) 


UNCLASSIFIED  -  For  Public  Release 


216 


UNCLASSIFIED 


UNCLASSIFIED 


DSTO-GD-0734 


UNCLASSIFIED  -  For  Public  Release 

i  Australian  Government 


Department  of  Defence 

Defence  Science  and 
Technology  Organisation 


Applying  FRA  as  part  of  an 
MBSE  methodology 


UNCLASSIFIED  -  For  Public  Release 


UNCLASSIFIED  -  For  Public  Release 


How  FRA  fits  in  with  MBSE 


Utilise  MBSE  capability  model  to  provide  context 

1 


Structured  analysis  framed  on 
functional  and  system  definition 


Likelihood:  MBSE  provides  structure 
to  elicit  and  store  the  likelihood 
information 

Consequence:  (quick  look)  Structure 
analysis  to  determine  flow  on  effects 
and  impact  to  mission  outcomes 
(model  traceability) 

(rigorous)  Perform  discrete 
simulations  of  different  risk  events 


Provide  structure  and  simple  scripting  to 
determine  overall  risk  level 


Figure  3:  Risk  management  process 


1 .  AS/NZS  ISO  31000:2009 
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Required  model  state 


•  Functional  decomposition  defined 

•  Functional  flows  modelled 

•  Information  flows  modelled  and  connected  to 
functions 


Required  for 
EFFBD 
representation 

_ J 


If  a  materiel  system  does  not  exist: 

■  Perform  risk  analysis  on  available  technologies  to 
perform  functions 

■  Identify  indicative  risk  areas  in  achieving 
functional  and  operational  outcomes  due  to 
technology  maturity  issues 

■  Repeat  FRA  when  the  materiel  system  is  known. 
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Model  elements  and  relationships 
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Information  Flows 


7 

Fuictjon2 

1 

Component  2 
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FRA  Process 

(Modified  FMEA  Process) 

1 .  Determine  objectives 

■  To  identify,  analyse  and  evaluate  risks  related  to  technical 
readiness 

2.  Identify  starting  points  for  analysis  (mode) 

3.  Identify  upstream  mechanisms  (causes) 

4.  Identify  downstream  effects  (impact  on  system 
performance  and  mission  outcomes) 

5.  Analyse  and  record  overall  risk  (trace  to  affected 
mission  outcomes) 
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3.  Consider  upstream  causes  of  functional  failure 

•  Use  tool  support  to  produce  report  on  “success  path” 

•  Start  from  chosen  function,  consider: 

■  “Triggered  by”  items:  Will  always  impact  flow 

■  “inputs”  items:  May  affect  quality  of  flow 

•  For  the  items  collected,  consider  the  other  functions 
they  are  “output  from”: 

■  If  multiple  “output  from”:  Redundancies  in  path 

•  Continue  backwards  through  the  success  path 


UNCLASSIFIED  -  For  Public  Release 


UNCLASSIFIED 


221 


DSTO-GD-0734 


UNCLASSIFIED 


UNCLASSIFIED  -  For  Public  Release 


Upstream  risk  patterns 


Redundancy  -  decreased  likelihood 


UNCLASSIFIED  -  For  Public  Release 


UNCLASSIFIED  -  For  Public  Release 


4.  Consider  downstream  effects  of  functional  failure 

•  Use  tool  support  to  produce  report  on  “success  path” 

•  For  each  function,  consider  the  items  it  “outputs" 

•  For  each  item,  consider: 

■  functions:  May  impact  flow,  but  also  need  to 
consider  if  other  functions  are  able  to  output  this  item 

■  “input  to”  functions:  May  affect  quality  of  flow 

•  Continue  forward  through  the  success  path 
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Downstream  risk  patterns 


Critical  path  -  significant  consequence 


Redundancy  -  decreased  consequence 


UNCLASSIFIED  -  For  Public  Release 


UNCLASSIFIED  -  For  Public  Release 


Analyse  and  record  resulting  risk 

•  Create  technical  risk  element  in  the  model,  related  to 
the  Function  /  Item  /  Link  analysed 

•  Record  risk  ratings  (likelihood,  consequence)  and 
mitigation  strategies 

•  Output  Technical  Risk  documentation  from  the  model 

•  Risk  can  result  in  a  design  decision  and  derived 
Requirement 
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5.  Analyse  and  record  resulting  risk 


Likelihood 

Consequence/Impact 

Minor 

Moderate 

Major 

More  Than 

Likely 

MEDIUM 

HIGH 

HIGH 

Less  Than 

Likely 

LOW 

MEDIUM 

HIGH 

Unlikely 

LOW 

LOW 

MEDIUM 

TRAH  Risk  Likelihood  /  Impact  Matrix 


1 .  Technical  Risk  Assessment  Handbook 
Requirements  Division.  DSTO,  2010 
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Example  -  Ground  Based  Air  Defence 
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Upstream  functional  traceability 

To  guide  the  analyst  in  understanding  the  potential 
influences  on  critical  functions 


What’s  the  likelihood  of  failure? 
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Upstream 
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Downstream 


1 


^ H 
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Benefits  of  methodology/  Conculsions 

Issues  with  current  practice 

FRA  Benefit 

TRA  templates  do  not  fit  every  type  of 
acquisition 

Focus  of  risk  analysis  is  on  a  model  of  the  capability 
of  Interest,  not  on  a  document  template. 

Documentation  is  derived  from  the  risk  analysis,  not 
the  other  way  around. 

Need  to  assume  a  materiel  solution 

FRA  can  be  applied  to  a  functional  description  of  a 
system  using  knowledge  of  available  technologies 
(pre-2'^^  pass)  and  is  repeated  for  physical  systems 
at  Pass. 

Quality  depends  on  the  skills  of 
individuals 

Provides  a  rigorous  process  to  assist  In  the  analysis 
of  whole  of  system  technical  risk 

Inadequate  analysis  of: 

System  integration  risk 

Risk  coupling 

Process  guides  analyst  through  the  potential 
influence  of  technologies  on  other  systems  and  sub¬ 
systems. 

Focus  is  on  potential  impact  of  integration  risk 

TRA  is  a  stand-alone  document 

Analysis  performed  in  and  risks  recorded  in  the 
same  model  OCD  and  FPS  definitions.  Completely 
traceable:  a  single  source  of  truth. 
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Additional  Benefits  /  conclusions 

Resulting  benefits  from  using  MBSE  for  risk  analysis: 

•  Capture  and  trace  risks  and  issues  to  mission 
objectives 

•  Capture  non-technical  risks/issues  (such  fitness-for- 
purpose) 

•  Can  extend  FRA  process  to  system  assessment 

•  Resulting  derived  requirements  can  be  traceable  back 
to  the  analysis  process 
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